深度学习已被广​​泛用于医学图像细分和其他方面。但是，现有的医学图像分割模型的性能受到获得足够数量的高质量数据的挑战的限制。为了克服限制，我们提出了一个新的视觉医学图像分割模型LVIT（语言符合视觉变压器）。在我们的模型中，引入了医学文本注释，以弥补图像数据的质量缺陷。此外，文本信息可以在一定程度上指导伪标签的产生，并进一步保证半监督学习中伪标签的质量。我们还提出了指数伪标签迭代机制（EPI），以帮助扩展LVIT和像素级注意模块（PLAM）的半监督版本，以保留图像的局部特征。在我们的模型中，LV（语言视觉）损失旨在直接使用文本信息监督未标记图像的培训。为了验证LVIT的性能，我们构建了包含病理图像，X射线等的多模式医学分割数据集（图像 +文本）。实验结果表明，我们提出的LVIT在完全和半监督条件下具有更好的分割性能。代码和数据集可在https://github.com/huanglizi/lvit上找到。

风险的准确器官（OAR）分割对于减少治疗后并发症的放射治疗至关重要。达人指南推荐头部和颈部（H＆N）区域的一套超过40桨的桨，然而，由于这项任务的可预测的禁止劳动力成本，大多数机构通过划定较小的桨子和忽视的少数，选择了大量简化的协议与其他桨相关的剂量分布。在这项工作中，我们提出了一种使用深度学习的新颖，自动化和高效的分层OAR分段（SOARS）系统，精确地描绘了一套全面的42 H＆N OAR。 SOARS将42桨分层进入锚，中级和小型和硬质子类别，通过神经结构搜索（NAS）原则，专门为每个类别提供神经网络架构。我们在内在机构中使用176名培训患者建立了SOAR模型，并在六个不同的机构中独立评估了1327名外部患者。对于每个机构评估，它始终如一地表现出其他最先进的方法至少3-5％的骰子得分（在其他度量的相对误差减少36％）。更重要的是，广泛的多用户研究明显证明，98％的SOARE预测只需要非常轻微或没有直接临床验收的修订（节省90％的辐射脑神经工作负载），并且它们的分割和剂量准确度在于或小于帧 - 用户的变化。这些调查结果证实了H＆N癌症放射疗法工作流OAR描绘过程的强烈临床适用性，提高了效率，全面性和质量。

3D描绘解剖结构是医学成像分析中的主要目标。在深度学习之前，统计形状模型施加解剖结构并产生高质量的表面是核心技术。在深度学习之前，统计形状模型施加解剖结构并产生高质量的表面是核心技术。今天完全卷积的网络（FCN），而主导，不提供这些功能。我们呈现深度隐式统计形状模型（幻像），一种划分的新方法，将卷积神经网络（CNNS）的表示力与SSM的稳健性结合。幻像使用深隐性表面表示来产生紧凑且描述性的形状潜空间，允许解剖学方差的统计模型。为了可靠地适应图像到图像，我们介绍了一种新颖的刚性和非刚性姿势估计管道，其被建模为Markov决策过程（MDP）。我们概述了一个培训制度，包括倒置的焦点培训和深度领域学习（MSL）的深刻实现。数据集关于病理肝脏分割任务的实验表明，幻灯片可以比三个领先的FCN模型更加强大，包括NNU-Net：将平均豪索轿车距离（HD）减少7.7-14.3毫米并改善最坏情况骰子索兰系数（DSC）达1.2-2.3％。更富豪地，直接反映临床部署方案的数据集上的交叉数据集实验表明，分别将平均DSC和HD分别改善平均DSC和HD，以及最坏情况的DSC 5.4-7.3％。这些改进超过了具有高质量表面的划分的任何益处。

Optical coherence tomography (OCT) captures cross-sectional data and is used for the screening, monitoring, and treatment planning of retinal diseases. Technological developments to increase the speed of acquisition often results in systems with a narrower spectral bandwidth, and hence a lower axial resolution. Traditionally, image-processing-based techniques have been utilized to reconstruct subsampled OCT data and more recently, deep-learning-based methods have been explored. In this study, we simulate reduced axial scan (A-scan) resolution by Gaussian windowing in the spectral domain and investigate the use of a learning-based approach for image feature reconstruction. In anticipation of the reduced resolution that accompanies wide-field OCT systems, we build upon super-resolution techniques to explore methods to better aid clinicians in their decision-making to improve patient outcomes, by reconstructing lost features using a pixel-to-pixel approach with an altered super-resolution generative adversarial network (SRGAN) architecture.

We introduce a new tool for stochastic convex optimization (SCO): a Reweighted Stochastic Query (ReSQue) estimator for the gradient of a function convolved with a (Gaussian) probability density. Combining ReSQue with recent advances in ball oracle acceleration [CJJJLST20, ACJJS21], we develop algorithms achieving state-of-the-art complexities for SCO in parallel and private settings. For a SCO objective constrained to the unit ball in $\mathbb{R}^d$, we obtain the following results (up to polylogarithmic factors). We give a parallel algorithm obtaining optimization error $\epsilon_{\text{opt}}$ with $d^{1/3}\epsilon_{\text{opt}}^{-2/3}$ gradient oracle query depth and $d^{1/3}\epsilon_{\text{opt}}^{-2/3} + \epsilon_{\text{opt}}^{-2}$ gradient queries in total, assuming access to a bounded-variance stochastic gradient estimator. For $\epsilon_{\text{opt}} \in [d^{-1}, d^{-1/4}]$, our algorithm matches the state-of-the-art oracle depth of [BJLLS19] while maintaining the optimal total work of stochastic gradient descent. We give an $(\epsilon_{\text{dp}}, \delta)$-differentially private algorithm which, given $n$ samples of Lipschitz loss functions, obtains near-optimal optimization error and makes $\min(n, n^2\epsilon_{\text{dp}}^2 d^{-1}) + \min(n^{4/3}\epsilon_{\text{dp}}^{1/3}, (nd)^{2/3}\epsilon_{\text{dp}}^{-1})$ queries to the gradients of these functions. In the regime $d \le n \epsilon_{\text{dp}}^{2}$, where privacy comes at no cost in terms of the optimal loss up to constants, our algorithm uses $n + (nd)^{2/3}\epsilon_{\text{dp}}^{-1}$ queries and improves recent advancements of [KLL21, AFKT21]. In the moderately low-dimensional setting $d \le \sqrt n \epsilon_{\text{dp}}^{3/2}$, our query complexity is near-linear.

Cashews are grown by over 3 million smallholders in more than 40 countries worldwide as a principal source of income. As the third largest cashew producer in Africa, Benin has nearly 200,000 smallholder cashew growers contributing 15% of the country's national export earnings. However, a lack of information on where and how cashew trees grow across the country hinders decision-making that could support increased cashew production and poverty alleviation. By leveraging 2.4-m Planet Basemaps and 0.5-m aerial imagery, newly developed deep learning algorithms, and large-scale ground truth datasets, we successfully produced the first national map of cashew in Benin and characterized the expansion of cashew plantations between 2015 and 2021. In particular, we developed a SpatioTemporal Classification with Attention (STCA) model to map the distribution of cashew plantations, which can fully capture texture information from discriminative time steps during a growing season. We further developed a Clustering Augmented Self-supervised Temporal Classification (CASTC) model to distinguish high-density versus low-density cashew plantations by automatic feature extraction and optimized clustering. Results show that the STCA model has an overall accuracy of 80% and the CASTC model achieved an overall accuracy of 77.9%. We found that the cashew area in Benin has doubled from 2015 to 2021 with 60% of new plantation development coming from cropland or fallow land, while encroachment of cashew plantations into protected areas has increased by 70%. Only half of cashew plantations were high-density in 2021, suggesting high potential for intensification. Our study illustrates the power of combining high-resolution remote sensing imagery and state-of-the-art deep learning algorithms to better understand tree crops in the heterogeneous smallholder landscape.

There are multiple scales of abstraction from which we can describe the same image, depending on whether we are focusing on fine-grained details or a more global attribute of the image. In brain mapping, learning to automatically parse images to build representations of both small-scale features (e.g., the presence of cells or blood vessels) and global properties of an image (e.g., which brain region the image comes from) is a crucial and open challenge. However, most existing datasets and benchmarks for neuroanatomy consider only a single downstream task at a time. To bridge this gap, we introduce a new dataset, annotations, and multiple downstream tasks that provide diverse ways to readout information about brain structure and architecture from the same image. Our multi-task neuroimaging benchmark (MTNeuro) is built on volumetric, micrometer-resolution X-ray microtomography images spanning a large thalamocortical section of mouse brain, encompassing multiple cortical and subcortical regions. We generated a number of different prediction challenges and evaluated several supervised and self-supervised models for brain-region prediction and pixel-level semantic segmentation of microstructures. Our experiments not only highlight the rich heterogeneity of this dataset, but also provide insights into how self-supervised approaches can be used to learn representations that capture multiple attributes of a single image and perform well on a variety of downstream tasks. Datasets, code, and pre-trained baseline models are provided at: https://mtneuro.github.io/ .

In this paper, we develop an efficient multi-scale network to predict action classes in partial videos in an end-to-end manner. Unlike most existing methods with offline feature generation, our method directly takes frames as input and further models motion evolution on two different temporal scales.Therefore, we solve the complexity problems of the two stages of modeling and the problem of insufficient temporal and spatial information of a single scale. Our proposed End-to-End MultiScale Network (E2EMSNet) is composed of two scales which are named segment scale and observed global scale. The segment scale leverages temporal difference over consecutive frames for finer motion patterns by supplying 2D convolutions. For observed global scale, a Long Short-Term Memory (LSTM) is incorporated to capture motion features of observed frames. Our model provides a simple and efficient modeling framework with a small computational cost. Our E2EMSNet is evaluated on three challenging datasets: BIT, HMDB51, and UCF101. The extensive experiments demonstrate the effectiveness of our method for action prediction in videos.

Off-policy evaluation (OPE) is a method for estimating the return of a target policy using some pre-collected observational data generated by a potentially different behavior policy. In some cases, there may be unmeasured variables that can confound the action-reward or action-next-state relationships, rendering many existing OPE approaches ineffective. This paper develops an instrumental variable (IV)-based method for consistent OPE in confounded Markov decision processes (MDPs). Similar to single-stage decision making, we show that IV enables us to correctly identify the target policy's value in infinite horizon settings as well. Furthermore, we propose an efficient and robust value estimator and illustrate its effectiveness through extensive simulations and analysis of real data from a world-leading short-video platform.

Patients take care of what their teeth will be like after the orthodontics. Orthodontists usually describe the expectation movement based on the original smile images, which is unconvincing. The growth of deep-learning generative models change this situation. It can visualize the outcome of orthodontic treatment and help patients foresee their future teeth and facial appearance. While previous studies mainly focus on 2D or 3D virtual treatment outcome (VTO) at a profile level, the problem of simulating treatment outcome at a frontal facial image is poorly explored. In this paper, we build an efficient and accurate system for simulating virtual teeth alignment effects in a frontal facial image. Our system takes a frontal face image of a patient with visible malpositioned teeth and the patient's 3D scanned teeth model as input, and progressively generates the visual results of the patient's teeth given the specific orthodontics planning steps from the doctor (i.e., the specification of translations and rotations of individual tooth). We design a multi-modal encoder-decoder based generative model to synthesize identity-preserving frontal facial images with aligned teeth. In addition, the original image color information is used to optimize the orthodontic outcomes, making the results more natural. We conduct extensive qualitative and clinical experiments and also a pilot study to validate our method.